Code Cleanup, Submodule Replacement, Error Handling Improvements & Deployment Enhancements #555
Conversation
…miscellaneous errors
Improve SNMP Error Code 10 Interpretation
… dependencies to `release_process.md`
…release_process.md`
…debug logging is enabled
Remove Stale Websocket Code
Print stack trace upon receiving bad encoded data from ACM only if debug logging is enabled
Removed `ppm_tim` service
Co-authored-by: Brandon Payne <[email protected]>
…or-ppm Updated mapfile reference in ppm*.properties files
…-sdw-depositor failing and updated env variables
S3 Deposit Submodule Removal and jpo-utils addition. Refactor of docker-compose integration.
Release Notes (2024 Q3 Release)
…ption-topic Set SDW_SUBSCRIPTION_TOPIC to 'topic.SDWDepositorInput'
ODE JSON Schema Updates
| @@ -17,3 +13,6 @@ | |||
| [submodule "qa/test-harness/ode-output-validator-library"] | |||
| path = qa/test-harness/ode-output-validator-library | |||
| url = https://github.com/usdot-jpo-ode/ode-output-validator-library.git | |||
| [submodule "jpo-utils"] | |||
| path = jpo-utils | |||
| url = https://github.com/usdot-jpo-ode/jpo-utils.git | |||
There was a problem hiding this comment.
@SaikrishnaBairamoni Do we need to add github actions to scan this new repo?
There was a problem hiding this comment.
@dan-du-car we did not setup GitHub actions for jpo-utils repo seems like this is new, I can create a story for this in our backlog. Thanks!
| @@ -28,7 +26,6 @@ jpo-ode-consumer-example.sonar.sources = src | |||
| jpo-ode-core.sonar.sources = src | |||
| jpo-ode-plugins.sonar.sources = src | |||
| jpo-ode-svcs.sonar.sources = src | |||
| jpo-s3-deposit.sonar.sources = src | |||
There was a problem hiding this comment.
If we are replacing the s3 depositor with jpo-utils, is it possible to add jpo-util to this scanner? @SaikrishnaBairamoni
There was a problem hiding this comment.
Yes, we can update the sonar properties as this PR changes are from CDOT fork I cannot make changes to this
| import us.dot.its.jpo.ode.wrapper.HttpClientFactory.HttpException; | ||
| import us.dot.its.jpo.ode.wrapper.HttpClientFactory.HttpResponse; | ||
|
|
||
| public class CASClient { |
There was a problem hiding this comment.
Why do we remove all the jpo-ode-core DDS components?
There was a problem hiding this comment.
The ODE previously used the DDS/WebSocket functionality to make deposits to the SDX. However, since the SDX no longer accepts deposits through this method, the DDS components are no longer needed.
| @@ -0,0 +1,31 @@ | |||
| name: Publish Java Package | |||
There was a problem hiding this comment.
What will be the approx. artifact size and where we are using this ? As our ODE GitHub org is on free plan we have artifact storage limit of 500mb.
There was a problem hiding this comment.
Approx:
jpo-ode-common.jar - 38 KB
jpo-ode-core.jar - 154 KB
jpo-ode-plugins.jar - 436 KB
jpo-ode-svcs.jar - 134 MB (large because it includes the Spring web stuff. We could probably make it much smaller by only releasing the normal jar instead of the Spring Boot uberjar)
I'm not certain, but this seems to be saying that the storage limits for maven packages don't apply to public repos:
https://docs.github.com/en/billing/managing-billing-for-github-packages/about-billing-for-github-packages
I am fairly certain that there is no time limit on Maven packages (which are not the same as Gitub Action build artifacts)
There was a problem hiding this comment.
I checked, and the normal jar for jpo-ode-svcs is approx 219KB, so if we changed the action to publish that, the total space would be less than 1MB for everything
There was a problem hiding this comment.
@iyourshaw Got it, I'm good with this change. This storage limits are applied for only private repositories. Thanks!
There was a problem hiding this comment.
@SaikrishnaBairamoni Here is the relevant PR that addresses this issue. We will work on getting this update merged ASAP.
Remove Sping Dependencies from Maven Jars
PR Details
Description
This PR includes the following changes:
Code Cleanup:
Submodule Updates:
Error Handling Enhancements:
Documentation and Configuration Updates:
Build and Deployment Improvements:
New Features:
Related Issue
No related GitHub issues.
Motivation and Context
The motivation behind these changes is to streamline the JPO-ODE project by removing outdated functionality, improving error handling, and simplifying deployment processes. By cleaning up unused services and submodules, the project becomes more maintainable, while enhanced error reporting increases transparency in debugging. The updates to the build and deployment pipeline, including resolving UID conflicts and automating JAR publishing, ensure smoother integration with modern workflows. Additionally, new features such as log offloading and support for source ASN1 bytes payloads improve operational efficiency and flexibility for users.
How Has This Been Tested?
These changes have been tested using unit tests, local deployments and the CDOT GCP deployments.
Types of changes
Checklist:
ODE Contributing Guide